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ABSTRACT 

An  asymptotic  analysis  is  presented  for  estimation  in  the  three 
parameter  first  order  autoregressive  model,  where  the  parameters  are 
the  mean,  autoregressive  coefficient,  and  variance  of  the  shocks. 

The  nearly  nonstationary  asymptotic  model  is  considered  wherein  the 
autoregressive  coefficient  tends  to  1  as  sample  size  tends  to 
Infinity.  Three  different  estimators  are  considered:  the  exact 
gaussian  maximum  likelihood  estimator,  the  conditional  maximum 
likelihood  or  least  squares  estimator,  and  some  "naive"  estimators. 
It  is  shown  that  the  estimators  converge  in  distribution  to 
analogous  estimators  for  a  continuous  time  Ornstein-Uhlenbeck 
process.  Simulation  results  show  that  the  MLE  has  smaller 
asymptotic  mean  squared  error  than  the  other  two,  and  that  the 
conditional  maximum  likelihood  estimator  gives  a  very  poor  estimator 
of  the  process  mean. 

Key  Words  and  Phrases:  likelihood  estimation,  autoregressive 
processes,  nearly  nonstationary  time  series,  Ornstein-Uhlenbeck 


process . 


1.  INTRODUCTION. 

Consider  a  sequence  of  statistical  experiments  with  observation 


vector  (yn(Q),  .  ..,  Yn(n))  given  by  a  three  parameter  AR(1)  process 


(1.1) 


[yn,lc+1)  -  „n]  -  rn[yn(k)  -  „n)  ♦  .„<*♦!>■ 


Of  if  • • • ,  n- 1 t 


The  shocks  4  (1),  . 4  (n)  are  assumed  l.l.d.  with  common 
n  n 


2  2 

distribution  independent  of  n,  and  E4n(l)  -  0  ,  E4n(l)  ■  aQ  <  •  .  We 


suppose  that  | T  (  <1  for  all  n  and  that  Yn(°)  ha*  the  stationary 


distribution  for  the  process.  The  parameters  r  and  u  will  be 

n  n 


allowed  to  vary  with  sample  size  (see  (1.2)  and  (1.3)  below). 

Suppose  that  the  statistician  models  the  process  as  Gaussian. 
Then  the  maximum  likelihood  estimate  (MLB)  of  the  parameter  vector 


(y  ,  T  ).  denoted  (u  ,  o„,  T),  is  a  solution  of  a  rather 

non  n  n  n 


complicated  system  of  equations.  Assuming  that  and  s  r( 

are  fixed,  then  one  can  show  that  the  MLB  is  asymptotically 

equivalent  to  a  simpler  estimator  obtained  by  maximizing  a 

conditional  likelihood.  The  MLB  maximizes  the  full  log  likelihood 

e  (u ,o2 ,r)  :»  log  f  2  -(y(l),  ...,  y(n)|y(0)) 
n  w  ,o  ,r 

+  i°g  t  2  (y(o) ) , 

M  M 

whereas  the  maximum  conditional  likelihood  estimator  (MCLE) 


maximizes  the  conditional  likelihood 


tn(v,o2,r)  :»  log  f  2  _(y(l),  ...,  y(n)|y(0)). 
n  y  ,o 


2 


~  ~2  m 

The  MCLE,  denoted  (jj  -o  -*  )  ,  Is  given  by  some  simple  formulae.  See 

n  n  n 

(3.12)  through  (3.15)  below.  Further  details  may  be  found  In  Fuller 
(1976),  pages  328-332. 

While  the  MLE  and  MCLE  will  be  nearly  the  same  with  high 
probability  for  "sufficiently  large  n“ ,  they  can  be  quite  different 
for  small  to  moderate  n.  Furthermore,  the  meaning  of  "large  n" 
depends  on  the  value  of  f .  If  r  is  close  to  1,  then  the  term 
log  f (y(0) )  -  (l/2)log[(l -r2)/a2)  -  ( 1-r2)  [y(0)2-m]/(2o2) 
has  a  more  pronounced  effect  on  the  log  likelihood,  and  a  much 
larger  value  of  n  is  required  before  the  classical  asymptotic 
results  are  useful.  As  many  real  series  exhibit  large  lag  one 
autocorrelation  (hence  r  near  1),  it  Is  worthwhile  to  Investigate 
the  MLE  and  MCLE  under  this  condition.  Furthermore,  one  is 
naturally  Interested  in  which  estimator  is  better,  or  if  some  other 
estimator  is  even  better  than  either  of  these.  One  would  conjecture 
that  the  MLE  is  better  than  the  MCLE,  and  we  present  results  below 
which  corroborate  this  conjecture. 

Recently,  there  has  been  much  Interest  in  "nearly 
nonstatlonary"  asymptotics  for  such  time  series  models.  See  e.g. 
Bobkoskl  (1983),  Chan  and  Wei  (1985),  and  Tsay  (1985).  For  the 
three  parameter  AR(1)  model,  this  corresponds  to  assuming  that 


where  PQ  and  vQ  are  fixed.  Since  rn  -»  1,  in  some  sense  the  process 
approaches  a  nonstationary  process  as  n-*“ .  The  rationale  for  the 
particular  foras  of  un  and  will  be  evident  from  the  following 
discussion. 

Define  a  continuous  time  "step  function”  process  Y  (t),  OStSl 


Yn(t)  n_1/2yn([nt]) . 

where  [•]  denotes  the  greatest  integer.  It  follows  from  (1.1)  that 

Y  satisfies  the  difference  equation 
n 


(1.4)  <iYn(k/n)  «  -0o[Yn(k/n)-i/o]4t  ♦  oQaWn(k/n) ,  osksn-l. 


Here,  4Yn(k/n)  :»  Yn( ( k+1 ) /n)  -  Yn(k/n)  is  a  forward  difference 
operator,  4t  :■  1/n,  and 


(1.5) 


-1  -1/2 

Wn<tJ  °0  n  lkml  ‘n<*> 


is  a  normalized  partial  sum  process.  Since  Wr  converges  weakly  to  a 
Wiener  process  W(t),  Odtll,  in  D[0,1J,  and  the  difference  operator  4 


converges  In  some  sense  to  a  differential  operator  d,  one  would 
expect  that  Yn  should  converge  to  the  solution  of  the  stochastic 
differential  equation 

(1.6)  dY(t)  «  -/J0[Y(t)-y0]dt  +  a  QdW  ( t )  , 

Y(0)  8  N(pq  «°q/  (  2/Sq  )  )  / 

Y ( 0 )  independent  of  (W(  t)  :0St31) , 

which  defines  an  Ornsteln-Uhlenbeck  process.  (Equality  in 

distribution  is  denoted  ».)  The  weak  convergence  of  YR  to  Y  follows 

from  Lemma  A.l.  in  the  Appendix. 

In  Section  3  this  weak  convergence  is  used  to  prove  convergence 

in  (joint)  distribution  of  the  MLE  (/>  ,o2,u  )  »  (n(l-r  ),o2,n 

n  n  n  n  n  n 

for  the  sequence  of  AR(1)  processes  given  by  (1.1),  (1.2),  and  (1.3) 
to  the  corresponding  MLE's  of  the  parameters  in  the 
Ornsteln-Uhlenbeck  process  given  in  (1.6).  See  Theorem  3.1  in 

Section  3.  The  MLE's  for  the  continuous  time  Ornsteln-Uhlenbeck 

-  *  2 

model  are  denoted  (p,u).  The  MLE  for  the  variance  parameter  is  oQ, 

l.e.  it  can  be  determined  exactly  (with  probability  one)  from  the 

2 

finite  sample  path  (Y(t):  OSt<l).  Indeed,  oQ  is  the  only  parameter 


which  is  consistently  estimable  from  the  sequence  of  AR(l) 
experiments . 

In  order  to  understand  this  phenomenon  and  to  define  the  MLE  in 

the  Ornstein-Uhlenbeck  model,  it  is  necessary  to  develop  the 

likelihood  (l.e.  Radon-Nlkodym  derivative  w.r.t.  some  dominating 

measure  on  path  space)  for  the  Ornstein-Uhlenbeck  model.  This  has 

been  done  by  Felgin  (1976)  for  the  situation  where  the  only  unknown 

parameter  is  pQ  and  Y(0)  is  taken  as  fixed  (l.e.  that  author  derives 

the  conditional  likelihood).  In  Section  2,  we  extend  those  results 

to  the  case  where  the  mean  uQ  is  also  unknown,  and  discuss  the 

"perfect"  estimabllity  of  the  variance  parameter  o^,  which  results 

from  mutual  singularity  of  the  Ornstein-Uhlenbeck  measures 

corresponding  to  different  variance  parameters. 

In  Theorem  3.2  in  Section  3  it  is  shown  that  the  MCLE 
~ 2  ~  ~  “2  -1/2“ 

(p  ,o,v  )  ■  (n(l-F  ),o  ,n  u  )  converges  in  distribution  to 
n  n  n  n  n  n 

2  “ 

(p ,o q,v) ,  where  p  and  v  denote  the  values  of  p  and  u  which  maximize 

the  conditional  likelihood  of  the  Ornstein-Uhlenbeck  observation 

given  the  starting  value  Y(0).  Theorem  3.3  gives  similar  limiting 

distribution  results  for  some  "naive"  estimators,  namely  the  sample 

lag  one  autocorrelation  rn  as  an  estimator  of  rn  *  (l-/B0/n),  a  crude 
2  2 

estimator  $n  of  and  the  sample  mean  of  the  yn(k)'s  as  an 


estimator  of  y  . 


While  these  results  give  representations  for  the  asymptotic 
distribution  of  the  estimators,  it  is  unfortunately  very  difficult 
to  carry  out  any  calculations  with  the  limiting  distributions. 
Bobkoskl  (1983)  gives  some  results  when  only  fiQ  is  unknown  and 
yn ( 0 ) *0 .  Of  course,  one  can  always  resort  to  Monte  Carlo,  as  we  do 
in  Section  4.  The  results  of  this  paper  do  provide  invariance 
principles  so  that  fixed  reference  distributions  can  be  developed 
for  samples  of  different  sizes,  even  if  computation  of  the  reference 
distributions  is  difficult.  Furthermore,  they  allow  one  to  obtain 
results  about  the  limiting  Ornstein-Uhlenbeck  case  by  simulating 
discrete  time  processes. 

Some  conclusions  and  conjectures  can  be  drawn  from  the 
simulation  results  presented  in  Section  4.  Firstly,  the  MLE  appears 
to  be  best  estimator  in  terms  of  mean  squared  error,  but  not 
significantly  so.  All  the  estimators  of  pQ  considered  are  biased 
upward,  especially  so  for  near  0.  (Hence,  the  corresponding 
estimators  of  rn  are  biased  downward,  especially  for  rn  near  1.) 

The  MCLE  estimator  of  the  mean  is  quite  bad,  much  worse  than  the 
sample  mean  or  MLE.  These  results  suggest  that  better  estimators  of 
pQ  may  exist  if  one  can  reduce  the  bias. 
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2.  THE  ORMSTEIN-UHLENBECK  PROCESS. 

In  this  section  we  derive  the  likelihood  for  a  continuous  time 
observation  {Y(t):  OStSl)  from  the  Ornstein-Uhlenbeck  process.  The 
derivation  is  standard  (see  scheme  (i)  on  p.  714  of  Feigin,  1976), 
so  it  will  only  be  sketched.  The  dominating  measure  is  a  Wiener 
process  measure  modified  to  account  for  starting  value  and  scale 
change.  Calculate  the  likelihood  ratio  of  the  finite  dimensional 
vector  ( Y( 0 ) ,  Y( 1/n) ,  Y(2/n),  ...,  Y(l))  under  the 
Ornstein-Uhlenbeck  measure  (numerator)  and  Wiener  measure 

lr 

(denominator)  and  let  n-*«  through  the  values  n  ■  2  . 

We  first  derive  the  conditional  likelihood  given  Y(0)  as  it  has 

a  simpler  fora  than  the  unconditional  likelihood.  The  latter  can 

then  be  obtained  by  modification  of  the  former.  Let  P(  •  |  Y(0),  u, 
2 

a  ,  p  )  be  the  Ornstein-Uhlenbeck  measure  on  path  space  C[0,1]  with 

mean  v,  scale  a,  and  drift  coefficient  p,  as  in  (1.6)  with 

2 

subscripts  deleted.  Let  Q(  •  |  Y(0),  o  )  denote  the  measure  of 
oW(t)  +  Y(0),  OstSl,  where  W  is  a  standard  Wiener  process.  For  the 
Ornstein-Uhlenbeck  process  we  have  the  following  integral 
representation  valid  for  any  t>s: 

rt 

(2.1)  Y  ( t )  -v  ■  expI-/J(t-s)  ]  [  Y  ( s )  -p  ]  +  a  exp[  -p(  t-x)  ]dW(x)  . 

J  s 

See  e.g.  Section  8.3  of  Arnold  (1974).  Thus,  the  sampled  process 
Y(0),  Y(l/n),  ...  an  AR(1)  process  with  mean  v,  autoregressive 


8 


coefficient  exp[-£/n],  and  shock  variance  a  ( l-exp[-2/»/n] )  .  Using 
this,  the  likelihood  ratio  can  be  shown  to  equal 

f  n. 


(2.2) 


I  n,  f  2/3/n  1 

exp  |  Tl0»ll-.Xp[-Wn]j 

n  rf  2/J/n  _  i  1  r  a\t  i  4  /  \  i  ‘ 

H^exprWnT  Xj 


-1 


2fl  ( l-exp[  -p/n]  ) 
oZ(  l-exp[-2,fl/n] )  i^O 


[Y(i/n)-y]aY(i/n) 


_  fln(  l-ggP.[_ig/.nl ).!  "y  [  y  (  i/n)  -u] 2  ( l/n)  }. 
o2  ( l-exp[  -2/s/n] )  i*0  J 

As  in  (9.3)  of  Feigin  (1976),  we  have 

(2.3)  ny1[JY(i/n)]2  l  a2. 

1*0 

2  k 

The  convergence  is  Q(  •  |  Y(0),  a  ) -almost  sure  if  n  »  2  and  k-*«, 
but  is  always  true  in  probability  by  a  Chebyshev  argument  with 
respect  to  either  P{  •  |  Y(0),  u,  a2,  p)  or  Q|  •  |  Y(0),  a2).  Some 
calculus  will  show  that  the  first  two  terms  in  the  exponent  in  (2.2) 
cancel  each  other.  After  computing  the  limits  of  the  third  and 
fourth  terms,  one  obtains  that  the  log  likelihood  is  equal  to 

1  2  1 

(2.4)  |  Y( 0 ) ,o  2 )  =  -  P  [  [ Y ( t ) -v ] dY { t )  -  -B-  f  [Y(t)-y}2dt. 

a  J0  2 a  J0 

2 

For  the  unconditional  likelihood,  let  P(  •  I  u,  a  ,  p)  denote 


the  Ornsteln-Uhlenbeck  Measure  whan  Y(0)  Is  given  Its  stationary 


distribution.  Let  Q{ 


a  )  be  the  measure  of  o[W(t)+Z],  OStSl, 


where  Z  Is  a  N(0,1)  random  variable  Independent  of  W(t),  0<t<l.  The 
likelihood  ratios  contain  extra  terms  in  the  exponent  from  the  ratio 
of  Initial  distributions.  These  are  easy  to  analyze  and  the 
likelihood  turns  out  to  be 


(2.5) 


2  1  Y(0) 

f(u,p\o  )  *  7log(2/J)  +  - * 

*  2a * 


n  r  r1  2  ^ 

-  \  \  [ Y { t ) -v ] dY ( t )  +  [Y(0)-v]“  I 

o  l  J0  J 

-  \  [ Y( t ) -v ] 2dt . 

2<J  J  0 

It  is  easy  to  solve  for  the  MCLE  for  p  and  v  from  (2.4).  The 


results  are 


(2.6) 


(Y(t)-YJ  dY(t) 
[Y(t)-Y]2  dt 


(2.7) 


Y  +  ( Y ( 1 )-Y(0) )/p, 


where 


2.8) 


v  =  r 


Y  ( t )  dt. 


The  MLE  also  exists,  but  is  not  so  easy  to  obtain.  One  can  solve 

2 

for  the  minimizer  over  u  of  t(v,p\o  )  for  each  fixed  p,  plug  that 


back:  In,  and  then  note  that  the  resulting  expression  as  a  function 
of  p  tends  to  -•  as  /J-*0  or  £-»•».  This  shows  that  the  MLE  exists. 


3 .  MAIN  THEOREMS . 


This  section  contains  tha  statements  and  proofs  of  the  claims 

that  the  parameter  estimates  for  the  nearly  nonatationary  AR(i) 

converge  to  their  analogues  for  the  Ornsteln-Uhlenbeck  process.  The 

first  theorem  concerns  the  HLE  and  the  second  concerns  the  MCLE. 

The  third  theorem  Is  about  some  "naive"  estimators. 

THEOREM  3.1.  Let  ( u  .a2.  T  )  be  the  MLE  of  (u  .  a2.  T  \  in 

n  n  n  non 

the  AR( 1 )  model  given  In  (1.1)  through  (1.3).  Let  (v,  p )  be  the  MLE 

o 

of  Pq)  in  the  Ornsteln-Uhlenbeck  model  In  (1.6)  when  o‘  is 

known .  Then 


(3.1) 


-1/2 
n  u 


n 


!  5 


n(l-r  ) 


n  J 


1/2 

PROOF ■  We  will  use  the  variables  n  u  in  place  of  u  and  l-p/n  In 
place  of  r.  Inessential  constants  In  the  log  likelihood  will  be 
dropped.  The  first  step  Is  to  eliminate  v  and  o  from  the 
likelihood  maximization  problem.  The  log  likelihood  can  be  written 

as 
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(3.2)  <n(n1/2u  ,o2,l-/J/n)  -  -( (n+l)/2)log  a 2  -  n/(2o2)s2 

+  ( 1/2 ) log  0  +(l/2)log(l-0/(2n) )  -  Bn(u)fl/a2  -  An(u)0 2/a2, 
where 


V*’1  : 


V"> 


*n  I 

(l/2)[-n"1[Yn(0)-y]2  +  J[Yn(k/n)-v]24t[ 
[Yn(0)-u]2  +  ^(Yn(k/n)-u]iiYn(lc/n)  . 


All  suuationt  In  this  proof  are  from  k«0  to  n-1,  unless  otherwise 

2 

Indicated.  For  any  fixed  values  of  o  and  fi, 


vn{p)  :m  (2^U-l/n)j’l[Yn(0)(l-/»/n)  +  Yn(l)  +/j^Yn(k/nMt] 


saxlslzes  <  over  u.  Note  that  sup  |  u  (/J)|  Is  bounded  in 
n  OS/J<«  n 

probability,  since  all  of  the  random  variables  appearing  in  the 

defining  expression  are  bounded  in  probability  by  Lemmas  A.l  and 

A.  2,  and  0Z.O .  Since  A_  and  B_  are  continuous  and  A_  is  bounded 

n  n  n 

below  by  a  function  of  Yn  only,  this  implies  that  V*>0,  3  C C2> 0, 


* 

3 


C _ ,  C.>0,  and  N  such  that  VnkN,  the  event 


En  :m  [  C1  +  C2fi  S  Bnll/n(fi))p  +  kn{vn{fi))p 
S  C 3fi  +  CAP2,  for  Vpkoj 


satisfies 


(3.3) 


P(*n)  *  1-*. 


For  each  fixed  value  of  p , 


(3.4) 


o2n(P)  [n/(n+l)]sn2  + 


[2/(n+l)][  Bn(vn(/J))/J  +  kn(un(p))p2  ] 


2  1/2"  2 

maximizes  over  o  the  function  *  (n  u(p),  a  ,  l-p/xi) ,  provided 

n  n 

*2  “2 

o  (/>)  >  0.  Note  that  on  the  event  F  .  a  ip)  >  0  for  all  n 
n  n  n 


sufficiently  large.  Also,  we  have 


(3.3) 


-  2n_1p0o0^yn(k/n)^WnOc/n)  +  n_1p2^Y2  (k/n)^t . 


The  first  tern  on  the  r.h.s.  of  (3.5)  -»  oQ  by  the  weak  law  of  large 
numbers,  while  the  other  two  terms  are  0p(n  1). 

With  a  little  algebra,  there  results 

(3.6)  2tn (n1/2un(fi),  oJ</>) ,  l-fl/n)  - 

-(n+l )  log  a2  (p)  +  log  p  +  log  (l-p/(2n)) 
n 

The  next  step  of  the  proof  consists  of  showing  that  p is 

bounded  away  from  0  and  •  in  probability.  Using  log  x  S  x-1,  Vx>0, 

on  the  event  B  we  have 
n 

(3.7)  2<n(n1/2wn(/J)  ,  oj(fl),  l-p/n)  +  (n+l ) log  s2  > 

(2/o2)[C3/J  +  CAp2]  +  log  p,  V/J«(0,2n). 

For  all  n  sufficiently,  the  expression  on  the  r.h.s.  of  (3.7) 

*  *  p  *  * 

achieves  a  maximum  at  some  pn  in  (0,2n),  and  pn  -*  p  ,  say.  When  Pn 
is  plugged  into  the  r.h.s.  of  (3.7),  the  resulting  expression 
converges  in  probability  to  a  constant.  Since  the  supremum  of  a 
lower  bound  on  the  likelihood  function  provides  a  lower  bound  on  the 


maximum  of  the  likelihood.  It  followa  that 

(3.8)  Vt>0,  3d,  N,  such  that  VnkN, 
p[2«n(n1/2Mn(in) ,  o*(pn).  l-fin/n)  +  (n+1 ) log  s2  k  m.j  i  1-*. 

Hence,  the  MLE  p Is  with  high  probability  In  the  set  of  /J«(0,2n) 
which  satisfy  the  Inequality  In  the  event  In  (3.8).  In  view  of 
(3.3)  and  (3.5),  we  may  restrict  attention  to  the  set  of  p'a 
satisfying  0S/)32n  and  for  some  constants  C5,  Cg>0,  and  a 

(3.9)  °n(0)  :m  -(n+1 )  log  Jl  +  +  Cg/»J  J  +  log  p  >  a. 

*  *  -1 

It  Is  easy  to  check  that  G_  Is  oaxlalzed  at  point  p  -*  C_  ,  that 

n  no 

G  (/»**)  -*  - ( C_+l )  -  log  C_,  and  that  G •’(/>)  Is  eventually  <  c  <  0 
n  n  5  o  n 

for  all  p,  where  c  Is  a  constant.  These  facts  Imply  that  there  Is  a 

constant  b>0  such  that  eventually  all  values  of  p  satisfying  (3.9) 

also  satisfy  psb.  How  Gn(fl)  -*  -[Cg+Cgfl]  +  0  as  IV4*  *  uniformly 

in  /s«(0,b],  and  the  limit  function  crosses  from  above  the  level  m  at 

*  * 

some  positive  value  larger  than  Pn  .  Por  0<pitor  Gn(0)  5  C  +  log  p 
for  all  sufficiently  large  n,  where  C  Is  some  constant,  so  G  must 


also  cross  ths  lsvsl  a  at  soas  point  in  ths  interval  (0>£n  )■ 
Hanes, 

(3.10)  V*>0,  3  a>0,  b>a,  N  such  that  VnfcN, 

P  [  0  exists  and  a  d  a  S  b  ]  2  1-4 . 
n  n 

It  now  follows  that  ths  MLE  (»>n,o2  #fln)  “  '°n^n^ 

exists  with  arbitrarily  high  probability  for  all  n  sufficiently 
large,  and  furtheraore  that  0 n  is  bounded  away  froa  0  and  •  in 
probability.  How  %(/»)  converges  in  probability  uniforaly  in 
/s«[0,b]  to 

(3.11)  v{0)  :»  [2  +  4  j  ~ 1  ( 0 )  ♦  Y(l)  ♦  />{  Y(t)  dtj  , 

and  0 2 { /J )  «  a2  ♦  0B(n-1)  $  a2,  uniforaly  in  p«[0,b].  Hence 
n  n  r  u 

e  (n1/2v  (0)  ,o2(/>)  ,l-/J/n)  +  [(n+l)/2]  log  o2  ♦  n/2  converges  in 
n  n  n  w 

probability  uniforaly  in  /)«(0,b]  to  t  (vn(fi) .0) ,  where 

t(v,0)  :»  (1/2)  log  0  -  B (u)0/a2  -K{v)02 /a2, 


Now  t(v,p)  is  the  likelihood  for  the  Ornsteln-Uhlenbeck  process 

2 

estimation  problem  (with  oQ  known,  of  course),  and  u{p)  is  clearly 

the  MLE  of  v  for  each  fixed  p.  It  follows  that  p  5  p,  the  MLE  of  p 

n 

in  the  Ornsteln-Uhlenbeck  setup.  The  proof  is  complete. 

# 


0.12) 


Now  consider  the  MCLE.  First,  define 

1  V 


rn0 


»t4oY“(t)- 


Then  the  MCLE's  are  given  by 
(3.13) 


n 


\  [y(t)-y0][y(t+i)-y0] 
\  [y(t)-y0]i 


(3.14) 


;n  -  Y0  *  *b^o>  - 

n(l-V 


(3.15) 


n 


The  corresponding  MCLE's  for  the  Ornsteln-Uhlenbeck  process  are 
given  in  (2.6)  through  (2.8).  The  following  theorem  can  be  proved 
more  simply  than  the  previous  one  by  simply  using  the  explicit 
formulae  for  the  estimators  and  the  results  in  the  Appendix. 


Theorem  3.2.  As  n-*« , 


f  -1/2“  1  f  “  1 

fn  M"!  r 

!  '<  !  2  !  •;  | 

t  n,1-’’n»J  l  0  J 


Finally,  we  consider  some  "naive”  estimators.  Let 


(3.16) 


(3.17) 


(3.18)  rn  - 


vnl-U 

v„  -  H7T  Jo  y(t’' 

J  tytt+D-y^tytt)-^] 

[ ;  l  tYit.n-Yji2;  ;  i  [yiti-Vo]2;  > 


refer  to  y  ,  s  ,  and  r  as  the  naive  estimators  of  u 
n  n  n  r 


r  ,  respectively. 

Theorem  3.3.  Let 


(3.19)  a 


^[Y(1)-Y(0)][Y(1)+Y(0)+2Y]  -  [ [ Y( t ) -Y]dY( t ) 
f[Y(t)-Y]*dt  ~~ 


Then  as  n-»« , 


r  y  i 

i  *  i 


-1/2- 


s2  I  2  I  o2 
Sn  I  I  0 


!  n(l-r_) 


PROOF .  We  will  assume  as  In  the  appendix  that  all  convergences  are 
taking  place  on  a  common  probability  space  so  that  we  may  use 
convergence  in  probability  rather  than  convergence  in  distribution, 

P 

and  -*  will  mean  -*  for  the  remainder  of  the  proof.  Now  it  is  clear 
from  Lemma  A.l  that 
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4.  MONTE  CARLO  RESULTS. 

Tables  1  through  3  present  the  results  of  a  simulation  study  of 

the  various  estimators.  The  simulation  program  used  the  IMSL 

subroutine  GGNML  to  generate  n+1  pseudorandom  variates  which  were 

used  to  construct  AR(1)  sample  paths  according  to  the  model  (1.1). 

We  considered  3  estimators  of  pQ  and  uQ  (the  naive,  MCLE,  and  MLE) 

2  “2 

and  4  estimators  of  oQ  (on  is  the  ordinary  sample  variance) .  All 

estimators  except  the  MLE  were  computed  directly  from  the  formulae. 

The  MLE  was  computed  by  a  Newton  type  algorithm  using  finite 

difference  approximations  to  the  derivatives  of  the  log  likelihood 

2 

function  as  a  function  of  p  with  v  and  a  substituted  out,  as  in  the 
proof  of  Theorem  3.1.  The  naive  estimator  was  used  as  starting 
value,  and  convergence  was  quite  fast,  requiring  on  the  average  less 
than  two  iterations  of  the  Newton  algorithm.  The  results  were 
compared  with  those  of  the  SAS  statistical  package  on  selected 
sample  paths  in  order  to  validate  the  program.  All  results  are 
based  on  25,000  Monte  Carlo  replications. 

The  results  indicate  that  the  MLE  is  the  best  of  the  estimators 
considered  in  terms  of  mean  squared  error,  although  not  by  much  in 
comparison  with  the  naive.  Two  surprising  results  emerge.  Firstly, 
all  estimators  of  pQ  are  badly  biased,  with  the  bias  becoming  worse 
as  p  becomes  smaller.  It  should  be  possible  to  find  improved 


estimators  of  pQ  by  "shrinking"  towards  0,  with  the  amount  of 
"shrinkage"  becoming  larger  as  say  the  sample  lag  one 
autocorrelation  becomes  larger.  The  bias  in  the  estimators  of  the 
other  parameters  was  negligible  compared  to  the  variance  and  so  is 
ommitted.  A  second  surprising  result  is  the  poor  performance  of  the 
MCLE  of  the  location  particularly  as  pQ  becomes  smaller.  This 

is  also  the  widely  used  least  squares  estimator  of  location.  The 
main  problem  here  is  the  term  (y(n)-y(O) )/p  (see  equation  (3.14)), 
which  severely  Inflates  the  variance.  Results  presented  by  Bobkoskl 
(1983)  Indicate  that  there  Is  some  probability  of  obtaining  p  close 
to  0  (it  may  even  be  negative,  which  is  why  p  was  not  used  as  the 
starting  value  for  the  iterative  calculation  of  the  MLE) .  This 
inaccuaracy  in  v  does  not  seem  to  present  a  problem  for  the  other 

-  -  2 

parameter  estimates  p  or  a  .As  the  MCLE  is  in  general  the  worst  of 
the  estimators,  we  suggest  that  one  use  either  the  naive  estimators 
or  the  full  MLE,  until  something  better  is  found. 


_•  *_§  • a  f  >.4  ' 


'tl.  t*.  it.  |Vi  .  i'*  |»|  4  ».l  M  i-»  L* 


TABLE  1.  SUMMARY  OF  SIMULATION  RESULTS 
FOR  ESTIMATORS  OF  0. . 


NOTES:  For  all  cases,  *'0“1  and  °o"1‘  E®tlaated  standard 

errors  are  shown  In  parentheses  next  to  the  figure. 


„  1 
*Q  1 

Estie-j 
ator  | 

Bias  | 

Mean  Squared  Error 

r 

4.38  ( .03) 

42.27  (.53) 

n 

5 

100 

*n 

4.37  (.03) 

43.58  (.54) 

'n 

4.48  ( .03) 

39.49  (.51) 

r 

4.55  ( .03) 

46.11  (.57) 

n 

S 

500 

*n 

4.55  (.03) 

47.89  (.59) 

4.22  (.03) 

42.87  ( .55) 

rn 

4.68  (.03) 

40.21  (.46) 

2 

100 

*n 

4.68  (.03) 

42.00  (.48) 

*n 

4.27  (.03) 

36.21  (.44) 

TABLE  2.  SUMMARY  OP  SIMULATION  RESULTS 
FOR  ESTIMATORS  OP  uQ . 


NOTES:  For  all  cases,  wQ»l  and  o**l.  Estimated  standard 

errors  are  shown  In  parentheses  next  to  the  figure. 

!  /j.  |  n  jEs*im  I  Mean  squared  Error  j 
I  o  I  I  ator  I  *  I 


Yn 

.032 

( .000) 

5 

100 

V 

n 

.376 

(.291) 

.029 

( .000) 

Yn 

.032 

( .000) 

5 

500 

un 

.  172 

( .061) 

wn 

.030 

( .000) 

Yn 

.  139 

( .001 ) 

2 

100 

V 

n 

286 

(257) 

vn 

.  125 

( .001) 

TABLE  3.  SUMMARY  OP  SIMULATION  RESULTS 
POR  ESTIMATORS  OP  o*. 

2 

NOTES:  For  all  cases,  uQ=l  and  oQ*l.  Estimated  standard 
errors  are  shown  in  parentheses  next  to  the  figure. 

|  *0  i  n  j  ^ator""|  Mean  S<*uared  Error 
|  5  |  10o|  all  |  .020  (.000) 

I  5  I  500 1  all  I  .0040  (.0003)  j 


APPENDIX. 


In  this  appendix  we  give  the  proofs  of  two  technical  lemmas. 

There  is  a  probability  space  carrying  probabilistic  replicas  of 

(*  (1),  ...»  *n(n))  for  «ach  n  and  a  Wiener  process  (W(t):OStsi) 

such  that  the  normalized  partial  sum  process  Wn(t)  satisfies 

sup  | W{ t )  -  W  ( t ) |  l  0, 

0<t<l 

p 

where  -»  denotes  convergence  in  probability.  See  Theorem  13.8  of 

Brelman  (1968).  We  assume  that  our  sequence  of  experiments  is 

defined  on  this  probability  space,  and  hereafter  deal  only  with 

convergence  in  probability.  The  results  then  transfer  back  to  the 

P  D 

original  probability  space  provided  one  replace  -*  with  .  Let  Y(t) 
denote  the  Ornstein-Uhlenbeck  process  given  by  the  stochastic 
differential  equation  in  (1.6),  and  Yn  the  normalized  AR(1)  process 
LEMMA  A . 1 . 


(A.l) 


sup  | V  ( t)  -  Y(t)|  l  0 
OStSl 


PROOF .  It  is  convenient  to  Introduce  a  Gaussian  step  function 
process  Yn(t)  by 


Y  ( (k+1 ) /n)  *  fjjk/n)  +  dW(k/n)  , 


Yn(t)  -  Yn(tntJ/n),  Yn(0)  -  (2/J0/[n(l-TP‘)])Y(0), 


where  4f(k/n)  -  f((k+l)/n)  -  f(k/n).  We  have  the  representation 
(A. 2)  Y  (t)  -  rCnt3y  (0)  +  .fl  r^[nt3“tnsI_1)dW(s) , 

AA  AA  AA  J  AA 


This  follows  from  the  usual  Inversion  formula  for  an  AR(1)  process 
e.g.  (2.3.3)  of  Fuller  (1976)).  Utilizing  the  analogous  formula 
(2.1)  for  the  Ornate ln-Uhlenbeck  process  we  have 


(A. 3) 


!Yn(t)  -  Y ( t ) |  S 


e  P°  -  rlnnt]  (2p0/{n(l-Tn2)))1/2\  |Y(0)  I 


♦  ■^V[ntI  -  ®  ^llj^  F"[nSldW(s)| 


rt  - r ns ] 

♦  e  I J  (rlJ  -e  u 
J0  n 


)  dW(s)| 


-1 


+  rn*|W(t)  -  W( [nt ] /n) | 


T.(t)  ♦  T  * ( t )  +  T  _ ( t )  +  T  . ( t ) ,  say. 


—  r  ng  i  P  o  ® 

Letting  fn(s)  : *  -  e  ,  one  can  show  via  elementary 

inequalities  that 

(A. 4)  0  S  fn(s)  *  C/n 

for  some  constant  C  (depending  on  pQ) .  From  this  and  (5.1.5)  of 
Arnold  (1974),  we  have 

r  rt  2122 

E  sup  I  f  ( s )  dW(s)|  S  4C  /n 

1  0<tSl  J0  1 

P  P 

and  hence  that  sup  Tn3(t)  5  0.  The  proofs  that  sup  Tnl(t)  -►  0  for 

1*1 ,2,4  are  even  easier. 

Now  consider 

(A. 5)  Yn(k/n)  -  Yn(k/n)  »  Y^[Yn(0)  -  Yn(0)] 

+  f  F<k"1"1,[ilWw(i/n)  -  aW(i/n)]. 
i£o  n  n 

Lindeberg's  central  limit  theorem  can  be  used  to  show  Yn(0)  5  Y(0), 
so  we  may  assume  that  our  probability  space  carries  a  version  of 
Yn(0)  such  that  |Yn(0)  -  Y ( 0 ) |  5  0,  and  then  the  first  term  on  the 
r.h.s.  of  (A. 5)  converges  to  0  in  probability,  uniformly  in  k, 
0<kSn.  For  the  second  term,  apply  partial  summation  to  see  that  it 


28 


is  equal  in  absolute  value  to 


(A. 6)  l%ltWn(k/n)  "  w<k/n)3  ~  "  x> 


£  ic-i 

i«n 


[W  { i/n)  -  W( i/n) ] | 


<  r~1(2  -  Fk)  sup  |W  (t)  -  W(t)| 
OS  til 


—  1  If 

Since  r  -»  1 ,  T  (2-F  )  is  bounded  uniformly  in  k  and  n,  so  it 
n  n  n 


follows  that  sup | Y  (t)  -  Y  ( t ) |  5  0. 

n  n 


LEMMA  A. 2 


(A. 7) 


PROOF .  We  have 


nr>l  P  f 1 

)  Y  (k/n)^W  (k/n)  £  Y(t)dW(t). 
k^O  n  n  J  0 


)  Y  (k/nMW  (k/n)  -  Y(1)W  (1) 

j^q  n  n  n  n  n 

+  (f”1  -  1)J  Yn(k/n)Wn(k/n) 

-  (i/2)o0F^V(i)  -  (i/2)o0r”1^  [JWn(k/n)]2 
Y( 1 ) W( 1 )  +  flj  Y(t)W(t)  dt  -  (1/2 )oQW2 ( 1 )  -  (1/2 )oQ . 


The  first  equality  Is  easily  checked  with  some  algebra.  The 

p 

convergence  of  the  first  three  terms  on  the  l.h.s.  of  the  -»  to  the 

p 

first  three  terms  on  the  r.h.s.  of  the  -»  Is  immediate  by  Lemma  A.l, 
and  the  fourth  by  the  weak  law  of  large  numbers.  By  Ito's  formula 
and  the  stochastic  differential  equation  for  Y  we  have 

d[Y(t)W(t)]  *  Y  ( t )  dW  { t )  -  /JY(  t)W(  t)dt  +  o  QW  ( t )  dW  ( t )  +  oQdt 
and  so 

|  Y  ( t )  dW  ( t )  *  Y(  1 )  W(  1 )  +  /ij  Y(t)W(t)  dt  -  ( 1/2  )oQW2  ( 1 )  -  oQ/ 2 
where  we  used  the  fact 

j1  W ( t ) dW { t )  -  (l/2)[  W2(l)  -  l]  , 

see  e.g.  Arnold  (1974),  page  76.  This  completes  the  proof. 

# 
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